Full presentation: Reasoning about machine ethics

We are entering an era where “intelligent” machines have unprecedented power over humans, including, sometimes, the literal power of life and death. Machine learning algorithms are increasingly used to classify humans as suspected terrorists or deserving recipients of financial aid. While military drones are currently flown by human operators, discussions about fully autonomous robotic soldiers are afoot. Once released on our streets, self-driving cars will almost certainly face circumstances where they just cannot avoid harming a human. It is thus imperative that the new generations of intelligent machines be not just efficient or functional but also ethical. If we can prove that a machine has behaved unethically, we can hold liable the operators and designers of the machine. But when can we consider a program to be ethical, and what are the design principles that we ought to follow to ensure ethical program behavior? The emerging field of machine ethics [2, 10, 1, 4] explores these questions. Machine ethics has received a significant amount of interest over the last decade, primarily from ethicists and artificial intelligence experts. Efforts here range from deliberations about the extent to which machines should be given ethical autonomy [1], to classification of AI agents into ethical categories [10], to logic-based methodologies that can guide engineering [5, 11], to concrete algorithms for ethical decision making [9]. Conspicuously missing from the area, however, are contributions by software engineering and programming language researchers. We believe that this is unfortunate for two reasons. First, the goal of machine ethics, at the end, is to guarantee that programs behave according to certain rigorous (moral and ethical) requirements. The area would thus seem to be a natural target for automated formal reasoning about programs. Secondly, machine ethics can be a source of interesting problems for the formal methods community. Most recent work in software analysis is motivated by applications in system software (such as operating system code). Machine ethics, in contrast, involves reasoning about knowledge, choices, obligations, etc. in high-level decision-making.